Machine Learning : Proceedings of the Thirteenth International Conference , 1996 Bias Plus Variance Decomposition forZero - One Loss

We present a bias-variance decomposition of expected misclassiication rate, the most commonly used loss function in supervised classiication learning. The bias-variance decomposition for quadratic loss functions is well known and serves as an important tool for analyzing learning algorithms, yet no decomposition was ooered for the more commonly used zero-one (misclassiication) loss functions until the recent work of Kong & Dietterich (1995) and Breiman (1996). Their decomposition suuers from some major shortcomings though (e.g., potentially negative variance), which our decomposition avoids. We show that, in practice, the naive frequency-based estimation of the decomposition terms is by itself biased and show how to correct for this bias. We illustrate the decomposition on various algorithms and datasets from the UCI repository.